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ABSTRACT 



An experimental verification of the continuous adjustment 
procedure considered by Sakrison is undertaken in this 
thesis. The adjustment procedure permits design of a 
filter system by continuous adjustment of k system para- 
meters so that the average error, weighted by a convex 
error criterion, is minimized. For experimental verifi- 
cation on typical analog equipment, this procedure is 
applied to the simple, yet typical, problem of separating 
a specified "voice" signal from broadband noise. The 
experimental work to determine the time required for 
convergence to a near optimum setting, the sensitivity of 
the system to the initial parameter settings, and the 
sensitivity of the system to the choice of parameters of 
the adjustment procedure showed that the adjustment proce- 
dure can be carried out on existing analog equipment. 
Further, since convergence times of well under 30 seconds 
can be obtained with this system and the system does 
perform satisfactorily over the required 30 db range in 
noise power density, it is applicable to use in an adaptive 
filter system for radio communications. 
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CHAPTER I 



INTRODUCTION 

A. INTRODUCTION 

An experimental verification of the continuous adjust- 



ment procedure considered by Sakrison[JLj is undertaken 
in this thesis. The adjustment procedure permits design 
of a filter system by continuous adjustment of k system 
parameters so that the average error, weighted by a convex 
error criterion, is minimized. For experimental verifica- 
tion on typical analog equipment this procedure is applied 
to the simple, yet, typical problem of separating a speci- 
fied "voice" signal from broadband noise. The separation 
of the desired signal from an input consisting of the 
desired signal plus noise is accomplished by linearly 
combining the outputs of k low pass filters for minimum 
weighted error. The experimental procedure is to determine 
the time required for convergence, the sensitivity of the 
system to the initial parameter settings, and the sensitiv- 
ity of the system to the choice of parameters of the 
adjustment procedure. 

* Numbers in brackets refer to items in the Bibliography. 

# Hqreafter referred to as "the adjustment procedure" . 
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B . BACKGROUND 



In order for the adjustment procedure to be applicable, 
the error criteripn must be convex; and, other than requir- 
ing that they be from related, bounded ergodic processes, 
no restriction is placed on the input and output. The form 
of the filter considered here is shown in Figure 1-1. It 
consists of k bounded, time invariant operations on the 
input. The outputs of these operations are to be linearly 
combined in such a manner as to optimize the system per- 
formance coefficients. If we let f. , denote the output 

1 , t 

til 

of the i such operation, f. must have a correlation 

1 , t 

with any linear combination of the other f . ^ which has a 

D/t 

magnitude less than one. 

A block diagram of the basic system is shown in Figure 
1-2 for a system with two parameters, i.e. k = 2. This 
system obtains its convergence by approximating the gradi- 
ent of the regression surface, MCx^^.x^) = E^W( e(t) )J, 
being searched and adjusting the values of x^ and X 2 to 
move the system toward the minimum error. Through the 
function c(t) a known plus and minus pertubation to the 
present setting of x^ and x^ is introduced. When the errors 



resulting from these plus and minus pertubations of the i 
parameter are measured, weighted by the chosen error 



. th 
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Figure 1 - 1 . Form of the filter (or predictor or model) to be designed.* 




x,(t) = x,( I) - c(T) Y| dT e| s -f|(t)[x,(t) +c(t)]-f 2 (t) x 2 (t) + d(t) 

xJt)*x (I) -/* -4^ YJTldT e i' 1 (t) - c (»)] - f 2 (t) x 2 (t) + d(t) 

2 2 •'i c(T) 2 

e 2 1 - f|( t) x,(t) - f 2 ( t) [x 2 0) +c 2 (t)] + d(t) 
e“ - -f,(t)x | (t)-f 2 (t)[x 2 (t)-c 2 (t)]+ d (t) 

figure 1-2. Diagram of the continuous adjustment procedure for two 
parameters, Xj and it 

B oth of these figures taken from [lj . 
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criterion, and subtracted, the resultant function, 
Y^(t)/c(t), is a random variable whose mean is a differ- 
ence approximation of the i component of the gradient in 
the direction of the optimum. The function a(t) is a mono- 
tonically decreasing function which is then used to weight 
the value of the gradient. This weighting function, a(t), 
approaches zero slowly enough that sufficient data is 
averaged to achieve the statistical optimum, yet approaches 
zero rapidly enough that the setting settles to a fixed 

value. The change of the parameter is then given by 

-t 

*1^) = x i (1) - Jtw Y i>> d y- • 

The form a(t) = K a /(at + b f* and c(t) = K c /(ct + d)^ 
satisfy the mathematical requirements for convergence if 
1/2 1 and d A >(l- < *' )/4. When c* = l, we also require 

that 0 <a/K a < 4K q . Here K Q is the largest number such 
that the regression surface gradient in the direction of 
the minimum is always greater than K Q times the distance 
to the minimum. The choice CY = 1 and S' ^ 1/4 gives a 
rate of convergence which cannot be exceeded in general 
for all processes and error functions meeting the previously 
stated requirements . The initial convergence depends 
critically on the adjustment coefficients a/K a , c/K c# cy , 
and S' . 



11 



A particular advantage of the adjustment procedure is 
that it is not restricted to the mean-square error criterion, 
and it can be carried out on analog equipment. In addition, 
if one desires to design analytically an optimum linear 
system, it is necessary to first measure the input and 
output statistics. After this either the Weiner-Hopf 
equation must be solved or, in the finite parameter case, 
a matrix must be inverted. An advantage offered by this 
adjustment procedure is that these two steps are replaced 
by a single, easily mechanized procedure. 

In a system where some error criterion other than the 
mean-square error is of interest, this method offers the 
advantage of being able to handle the broad class of convex 
error criterion. This latter advantage disappears when the 
input and output are jointly gaussian since for this case 
the optimum filter for the mean-square criterion is 
the optimum filter for any convex error criterion (XI - 
For the case in which the signals involved are non-gaussian 
and a non-linear filter is desired, the adjustment proce- 
dure is again suitable. For the situation in which the 
noise is quasi-stationary (nearly constant over long periods 
with respect to the time required for the procedure’ to 
settle to a near optimum setting) , this procedure would 
also be useful in an adaptive configuration. 
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It is to be noted that this system requires that the 
desired output signal be available. In this adjustment 
procedure duplication of the conditions under which the 
filter will have to operate (including real time data in 
the operational environment) spares one the necessity of 
having to measure the statistical character of the input 
and output. In a design based on mathematical calculation 
of the optimum coefficients for which the statistics are 
unknown, we must have available the desired output for a 
fairly long period of time in order to measure the appro- 
priate statistics. Periodic application of the adjustment 
procedure to correct for slowly changing statistics 
requires that the desired output be available at the 
receiver for only a small percentage of the time. More- 
over, in certain situations of interest the procedure 
can be carried out without the desired signal QlJ . It 
is sufficient to know that the previously mentioned 
constraints are met. 

C. TWO ADAPTIVE CONFIGURATIONS 



There are two particular problems which are of interest 
to mobile military units for which this procedure might 
provide a suitable solution in an adaptive configuration. 
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Let us consider the problem of radio communications in 
the presence of atmospheric noise. Crichlow et al. [J^Zl 
have published data on actual radio noise measurements at 
several locations. For a location near Front Royal, Va. 
at a frequency of 2.18 Me. it was found that the four hour 
median of the noise level varied over a range of about 30 
db during a mean 24 hour period. Deviations of up to 10 db 
of the one hour mean from the four hour mean were also 
recorded. However, it was found that when the short time 
variations in noise power are averaged over a long period 
(several minutes) the average noise power level is nearly 
constant for any given hour with variations seldom more 
than 2 or 3 db except during sunrise and sunset periods. 
Since the noise is quasi-stationary for a period of about 
an hour, it would be appropriate to apply the adjustment 
procedure in an adaptive fashion in a radio communication 
link if the procedure leads to convergence within a period 
of less than 30 seconds (convergence times much less than 
this have been obtained in the course of the present 
investigation) . Such an adaptive system could be used to 
achieve either a fixed signal to noise ratio while varying 
the transmitter power or a maximum signal to noise ratio 
with fixed transmitter power. 
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A primary consideration for operation of some military 
communications systems is the concealment of the location 
of the transmitting unit from an enemy. With a capability 
of precisely adjusting the transmitter power for a desired 
signal to noise ratio at the receiver (to be accomplished 
by a feedback signal to the transmitting unit from the 
receiving unit whose location need not be concealed) we 
reduce the probability of enemy location of the transmit- 
ting site compared with a situation in which the same 
transmitter power is used for all noise conditions. 

In the case of filtering an audio signal from varying 
noise, the use of statistical methods are possibly of 
marginal value since they result in a rather modest improve- 
ment over filters designed by intuitive methods (this point 
will be discussed later) . This example was chosen for the 
present work, however, because it is fully adequate for the 
purpose of evaluating the adjustment procedure and could 
be easily implemented with available equipment. 

One example of interest for military application in 
which substantial improvement is possible is in the field 
of magnetic detection devices. In particular, we might be 
interested in detecting submarines by aircraft using 
magnetic airborne detection (MAD) equipment. The problem 
here is that the background magnetic signature of the earth 
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changes with time as well as with geographic location. In 
such cases an aircraft might easily encounter several 
different backgrounds on one search flight. Different 
areas might be searched on different days, also causing a 
change in the background noise spectrum. If we assume an 
aircraft speed of 200 knots, we might anticipate that the 
signature of a submarine would last for no longer than five 
or six seconds as a pulse introduced into the background. 
Since this time is very short compared with the time over 
which the background noise remains approximately constant 
(an hour or so) , the adjustment procedure could be efficient- 
ly applied here. 

If one conducts experiments with friendly submarines, 
it is possible to predict the pulse shape to some accuracy. 
Thus, it will be possible to have available a desired 
signal for use in adapting the system to the needs at hand. 

A matched filter arrangement can be improved if there is 
a means of adjusting the filter to varying noise spectra, 
which is the case considered here. 

D. A PROPOSED ADAPTIVE SYSTEM 



For purposes of checking the convergence properties of 
the adjustment procedure, it was decided to employ the 
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procedure in a self optimizing or adaptive filter for 
separating a speech signal from broadband noise of slowly 
varying level. Specifically, an adaptive system which 
might have application in a radio communication network 
between ships was selected. 

Pant |jd describes the long-term average of speech 
as being characterized by a 12 db per octave decay over the 
audio spectrum. He also shows some curves of the actual 
spectrum which suggest that restricting the circuit usage 
to certain types of messages (i.e. a tactical voice 
circuit) would result in a power density spectrum which 
would follow the 12 db curve on the average, but which 
would have significant plus and minus perturbations from 
this mean. With such a desired output spectrum, which is 
essentially a staircase function with steps of uneven 
height following the mean of 12 db per octave slope in the 
frequency domain, it was decided that an investigation 
should be made using an adaptive filter system consisting 
of k low pass filters spaced linearly throughout the useful 
portion of the audio band (250 to 2500 cps contains most 
of the useful intelligibility) . The adaptive system would 
then compare the desired output with that actually being 
received and optimize the coefficients of the k filters 
for minimum mean-square error. 
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In order to show that an adaptive system has some 
merit in the use outlined above, let us examine the improve- 
ment in signal- to-noise ratio of the adaptive filter over 
a fixed filter. We will consider that we have available 
ideal bandpass filters with adjustable multiplicative 
coefficients (k^) . With N such filters linearly spaced 
over that portion of the audio band containing the useful 
intelligibility, the transfer function of such a filter is 

k iH U _l ( M- «i> ~ M- , 

where U_^(x-a) is a unit step function which is 0 for 

x < a and 1 for x > a and co = 2n-(250 + , i = 0, 1, 

1 N 

2,...,N. Note that N ideal bandpass filters can be obtained 
from a linear combination of N + 1 ideal low pass filters. 

We then consider the spectrum of the desired signal 
as being of the form 

M 

a j C u -i(!^'- Wj) - U_ 1 (|co|- COj _ 1 )] 



where . = co for i = j and N = M and the a . are per- 
3 i 3 

K 

turbations from the form ■ . for the aj 

(3 co + 628) 2 3 

evaluated at 6_> j, and we assume that the a^ are constant 

2*2250 

over the increment Aco = . We assume that the 

M 

signal is subjected to additive noise of the same form as 



the signal with N Q j replacing the a_. . We also assume that 
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the N . are constant across the increment Aco and that the 
cross correlation between the desired signal and the noise 
is zero. 

With these assumptions the form of the mean-square 
error becomes 



e = 



where 



1 - Cs Dl-H(s)J [l-H(-s)J ds + -i -7 1 



2n i J o. 

'J* 

M 



N 

2nj ^ 



H (s) 



ds 



S - E a -i 60.) - U_ x (| 6jj - CJ )] 

H(s) = ^ k. [u.^iq- co i ) - U_ 1 (|6jj- OD i _ 1 )] 



At 



H -E*C J &- 1 (I“|- "0 - D . 1 (l"|- CO a 
j*‘ ■ 

If we let M = N for simplicity, the mean-square error is 



>2 = 



'C- I 



Then by taking partial derivatives we find the optimum values 

^i — sc 

of k. , k. . = , and the minimum value of e z , 

i i opt a^+Noi 



N 






min 



_ Ago " N 1 a i N oi 

’ £/ a i +N oi 



N 

2 N <,i k i 



Aco 

~ ~ — l_x iN oi^i opt . 

■<* / 



As a check note that when N o; ^ = 0, k^ = 1 and when N 0 ^ = <*> , 
k^ = 0 for all i. 

If we make a stepwise approximation to the mean speech 

power spectrum described by Fant with M = N = 10 and 

A Co = 2^2250, we have 

M 
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a 1 = 27.0 


a 5 = 3.31 


a^ — 1 . 26 




a 2 = 12.6 


a 6 = 2.46 






a 3 = 7.08 


a^ — - 1 * 82 






a 4 = 4.74 


cig — 1*50 




• 



We now find the optimum k^ for N c ^ = 5.0 for all i. We now 

consider that these k- . are those which would have been 

i opt 

obtained if this particular type filter had been designed 
for the average condition. We further assume that the 



a i 


of the signal 


are 




a l 


= 30 


a 5 = 4 


a 9 = 1 


a 2 


= 13 


a 6 = 2 


a 10 = 1 


a 3 


= 4 


— 2 




a 4 


= 4 


a 8 = 1 





These values are chosen to obtain about the same input signal 
to noise ratio as was available to the mean design. The 
performance of a fixed filter compared with the optimum 
filter is shown in Table I. 



TABLE I 



COMPARISON OP THE OPTIMUM FILTER WITH A FIXED FILTER 



N o 


e 2 • 
e min 


e2 f ixed 


% improve 


snr in 


snr out 


sm^/snr^ 


.5 


4.11 


9.94 


142.0 


12.4 


15.81 


1.28 


5.0 


19.62 


20.2 


2.8 


1.24 


2.96 


2.39 


50.0 


46.90 


122.8 


162.0 


.124 


.475 


3.83 



Thus, we see that significant improvements are possible with 



an adaptive system. 
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Having seen that for the case of the ideal bandpass 
filter it is possible to obtain improvement over a 

fixed system, we next turn to the problem of actually 
checking the convergence of such a system. Considerations 
to be dealt with below restrict the error criterion to 
either an absolute value or a mean-square criterion. Since 
the mean-square criterion is more amenable to analytic 
treatment, we have in this case some way of knowing if the 
results given by the system are near the optimum. Hence, 
the mean-square error criterion was chosen. 

The spectrum of the desired voice signal is of second 
order. In order to obtain sharp resolution in our process 
we must then use filters of at least third order. To 
reduce the analytic treatment to reasonable proportions 
and to use the tables available in Newton, Gould, and Kaiser 
U5] to evaluate the error, third order filters were chosen. 
These tables provide an easy means of evaluating the mean- 
square error as a function of the variable parameters, k^. 
Having the mean-square error in terms of the k^ permits us 
to find the optimum simply by solving the set of equations 
resulting from setting the partial derivatives with respect 
to each of the k^ equal to zero. It is then possible to 
calculate the minimum mean-square error for comparison with 
that found experimentally. 



21 



CHAPTER II 



PROCEDURE 



A. THE ANALOG COMPUTER 



One of the advantages claimed by Sakrison for his 
adjustment procedure is that it can be realized on simple 
analog computing facilities [lj . A Philbrick analog 
computer was made available by the Dynamic Analysis and 
Control Labratory so that this procedure might be checked. 
This computer consists of 20 universal linear operator 
units (K5U) , 4 stabilized multiply/divide units (K5M) , 6 
unstabilized multiply /divide units (MU/DV) , and an eight 
channel oscilloscope for output read-out. 

The K5U units permit operations of the form 



e out^ e o + 10 ^JH e inJ 



(t) 



or 



/ 

4 - 



'out 



(t) = e Q + 10 m fy~| a i e -ln. (t)dt 



-11.10 ^ a i ^ 

m = 0, 1, 2, 



-50.0 ^ e Q ^ 



*-/ 



These units have a frequency response which is flat to 



11.10 

3 

50.0 



beyond 6000 cps. 
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M 













The basic equation for the operation of the K5M units 
e 1 (t)e 2 (t) 

is e (t) = . These K5M units also permit the 

out e 3 (t) 

addition of a constant to one of the two multiplication 

inputs, to the division input, or to the output. Figure 

II-l shows the output amplitude response versus frequency 

for the K5M units. In addition it should be noted that 

the waveform was badly distorted above 1500 cps. 

The MU/DV units have a response characteristic similar 

to that of the K5M units. In the multiplication mode 

their output is of the form e ut . (t) = e l/ t ^ e 2 

25 

drift in these units is of such a magnitude that they must 
be checked at least once an hour to insure that they meet 
the accuracy required by the problem at hand. 

The maximum range without saturation for all these 
units is i 50 volts. There is an inherent noise level of 
about 0.035 volts rms in the multiplication units, and a 
noise level of less than 1 mv. in the K5U units. The above 
noise levels are referred to the input so that for high 
gains the noise is also amplified. All of these units are 
capable of driving up to four other units. 
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B. EXPERIMENTAL PROCEDURE 



Experimental work was undertaken to insure that the 
adjustment procedure would lead to system parameter con- 
vergence on the available analog equipment. In particular, 
it was desired to know if the system parameters would 
converge within a time suitable for the application proposed 
and if the adjustment procedure was critically dependent on 
any of the adjustment parameters. Sakrison has shown that 
for a related discrete adjustment process the settings of 
the parameters of the adjustment process are quite impor- 
tant [jf] . Certain settings of the adjustment parameters 
in the discrete case led to violent oscillations for a 
prolonged period of time before settling down, and for 
other settings the behavior was quite overdamped and 
convergence did not occur within a reasonable period of 
time . 

The use of a mean-square error criterion in this 
experimental work has been mentioned above. The analytical 
convenience offered by this choice is obvious; however, our 
freedom of choice is not as great as might be implied since 
the number of available multipliers also restrict the 
critera which may be considered. Better appreciation of 
the latter restriction is gained by examining the schematic 
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of three convex error criteria for adjustment of only two 
filter gains. 

Figure II-2 shows the schematic diagram for the abso- 
lute value criterion. In this particular arrangement we 
have simplified the problem by assuming that a(t) = c(t). 

It is seen from this figure that two multipliers are 
necessary for each parameter to be adjusted. 

The arrangement shown in Figure II-3 is for the square 
error criterion. It will be noted that fewer linear oper- 
ator units are required for this configuration than for the 
absolute value arrangement, but that the number of multi- 
pliers remains at two per parameter being adjusted. 

In order to meet the requirements of a fourth power 
error criterion. Figure II-4, eleven multiplier^ or 4 per 
parameter being adjusted plus one common to all the adjust- 
ment parameters, are needed. For the analog computer 
available for this work the constraint imposed by the number 
of multipliers available quickly limits us to either the 
absolute value or to the square error criterion. The choice 
of the latter was then made on the basis of computational 
convenience. 

It was decided that extensive studies of the perform- 
ance of the adjustment procedure for a filter system 
consisting of two filters should be conducted. On 
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Figure II-2. Schematic diagram for absolute value error 
criterion. 
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Figure II-3. Schematic diagram for square error criterion. 
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Figure II-4. Schematic diacrrara tov fourth order error c v ‘ te 
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f: ; , ... • II- 'i . Computer program for experimental study of 
two filter system. 
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completion of these tests^ examination of a four parameter 
should be conducted. In the four parameter case, analyti- 
cal determination of the optimum values of the coefficients 
is difficult since available tables do not permit evalua- 
tion of twelfth and fourteenth order integrals. For 
purposes of this experiment, however, it was sufficient 
to show consistent values of the mean-square error from the 
four parameter system optimized by the adjustment procedure. 

In the four parameter case eight K5U units are 
required for auxiliary uses associated with signal shaping, 
function generation, and summing operations common to all 
filters. Since one integrator (K5U) is required for each 
parameter adjusted and simulation of third order filters on 
the analog computer requires the use of three K5U units for 
each filter, there are an insufficient number of computing 
elements available for the four parameter case entirely 
simulated on the computer. Therefore, in order to obtain 
consistent results between the two and the four parameter 
case, filters formed by RC networks in the input and feed- 
back paths of stabalized operational amplifiers were used. 

The form of the transfer function of the filters is 
a 

0 

H(s) = . The time constants of the filters 

(a^s+1) ^ (a 2 S+l) 

used in the experimental work are shown in Table II. 
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Filters 2 and 3 were used for the two filter studies. The 
weightings of their outputs are the parameters to be opti- 
mized, and these are hereafter called k-^ and k 2 for filters 
2 and 3 respectively. 

TABLE II 

TIME CONSTANTS OF FILTERS USED FOR EXPERIMENTAL WORK 



Filter number 


a 0 




a l 


a 2 


1 


.239 




.00130 


.00635 


2 


.187 




.000424 


.000213 


3 


.185 




.000252 


.000130 


4 


.140 




.000180 


.0000916 


Figure II-5 


shows 


the computer program 


for the experi 


mental work with 


two filters. 


This program 


follows the 


diagram shown in 


Figure 


II-3 


except for the 


amplifiers 


following the filters. 


These 


amplifiers are necessary, 



in this case, to increase the level of the waveform at 
that point to avoid computer noise. In Figure II-5 
capital letters are gains of the associated K5U units , 
underlined capital letters are initial conditions, and 
small letters denote the time function at the output of the 

associated unit. From Figure II-5 it is seen that k.^ = l , 

BUk * tt 50e-^ 1 ^ 

k 2 = 2 , and e 2 = , where the k^ are the parameters 

25 E 

being optimized for minimum e^. 
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The desired output, S, is obtained by passing the output 

of a white noise generator through a filter of the form 
K 

-j~. . — - 7 2~ • This gives the desired approximation of 

\ 3 CO +040 ) 

a mean speech spectrum. 

To the desired output was added white noise from 

another generator. A separate noise generator was used to 

insure minimum correlation between the desired output and 

the noise. This sum, when used as the input to the filter 

system, gave a good approximation to speech distorted by 

the type of noise likely from atmospheric sources. 

The values of both the signal power density, S Q , and 

the noise power density, N Q , were determined by assuming 

that they were from white noise sources having a flat 

power density spectrum beyond the maximum frequency of 

interest. (Later measurements of the power density spectra 

of the two sources validated this assumption.) For the 

purpose of measuring N 0 , the output from the white noise 

generator was passed through a single pole low pass filter 

having a half-power cut-off frequency about equal to the 

highest value to be used in any of the test filters. The 

resultant waveform was squared and averaged by integrating 

the squared output for one second. The tables of Newton, 

Gould and Kaiser [j>J were then used to determine the value 

of N . 
o 
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The procedure used to evaluate S Q was the same as 
that above except that the white noise was shaped prior 
to being inserted into the single pole low pass filter. 

The same low pass filter was used for both cases. 

The function a(t) was generated by inserting a ramp 
of the form A’t + 2 into the division input of a K5M unit 
and then multiplying the output of the K5M unit by the inte- 
grator gain, D, so that the form of a(t) was • It 

was found that constants less than 2 in the ramp function 
caused internal saturation in the K5M units and that 
inconsistent results were obtained when such saturation 
occurred. Since the lowest constant value was desired to 
derive maximum benefit from a(t) as a time function, the 
value 2 was chosen. 

The experimental value of the mean-square error was 
found by squaring the error signal and integrating this 
value for a fixed time period. Time periods of 0.1, 1, 
and 10 seconds were used depending on the rate of change 
of the error. 

For purposes of this study convergence time is defined 
as the time elapsed from activation of the system until the 
last time that the mean-square error falls to within 10% 
of its minimum value. That region containing all those 
points having their mean-square error within 10% of the 
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minimum value is hereafter called the convergence area. 
Although this definition of convergence is quite arbitrary, 
it does give a fairly concise definition of the time taken 
to reach near optimum performance. A value less than 10% 
would lead to a greater variance in the convergence time 
due to the decreased gradient of the regression surface 
in the vicinity of the optimum. 

Convergence times were obtained by stop watch measure- 
ments while observing an oscilloscope displaying k-^ versus 
^2 and noting the time to enter the convergence area which 
had been drawn on the face of the CRT. 

As a matter of convenience, the term "feedback loop" 
is defined as referring to that portion of the system from 
the output of the error summing K5U, e(t) in Figure II-5, 
through multiplier 3 (or 4) and integrator 5 (or 6) to multi- 
plier 1 (or 2) . 

The results desired from the experimental work were: 

1) the effect of the initial settings of the k^ on conver- 
gence time, 2) the effect of feedback loop integrator gain, 
D, on convergence time with A’ fixed (note that the same 
gain setting is used for all feedback loop integrators) , 

3) the effect of changes of A’ on convergence time for 
D fixed, 4) the range of noise power density over which 
it is possible to obtain convergence with S Q fixed. 
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5) the values to which the k^ converge, and 6) any other 
information concerning the use of analog equipment to 
implement this adjustment procedure. 

The desired form of the experimental data was: 1) 
numerical time values where convergence time was involved, 

2) recordings of the k^ and e? as functions of time, and 

3) numerical data on the final values of the k^. The 
recorded data were taken on a Sanborn two channel recorder. 
Final values of the k^ were recorded at the end of 50 
seconds for A' - 1. 
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CHAPTER III 



RESULTS 



A. THEORETICAL RESULTS 

In order to check on the accuracy of the convergence 
procedure, studies were made of the theoretical values of 
the optimum k^ and the minimum mean-square error for the 
two filter case. The problem to be solved was in the form 




jC<i 



+ J N H(s)H(-s)ds 

~r ° 



(i) 



where 
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H(s) = k l a 0.2 

< a 1.2 s+1) <a 2 >2 s+l) 




S = Power density spectrum of the desired signal 
before shaping 



N Q = Power density spectrum of noise 

a- .= i^ time constant of j ^ filter (see Table II) 
1 # J 



We see that a sixth and an eighth order integral ( in 
the Newton, Gould, and Kaiser sense [¥] ) must be evaluated 



to obtain the desired result. Since such calculations are 



exceedingly long and many similar calculations were desired 
for purposes of properly evaluating the proposed experi- 
mental work, the use of a digital computer was expedient. 
Time was made available by the Civil Engineering Systems 
Labratory on their IBM 1620 computer for this purpose. 

A program was written which used the time constants 
of each of the filters, the signal power density, the 
noise power density, the experimental values of the k^, 
and the time constant of the desired signal shaping filter 
to compute the necessary values of the experimental mean- 
square error, following the appropriate tables in Newton, 
Gould, and Kaiser [V] ♦ It can be shown that solution of 
(1) following these tables leads to an equation of the form 

e^ = C 0 +C 1 k 1 +C 2 k 2 +C 3 k 1 k 2 +C 4 k 1 2 +C5k 2 2 (2) 

where the are constants. The equations necessary for the 
computation of the optimum values of the k^ and the minimum 
mean-square error were easily obtained from (2) . These 
equations were inserted in the digital computer program so 
that for any given set of values of signal power density, 
noise power density, and experimental k i# it is possible 
to compute 1) the theoretical value of the mean-square 
error based on the experimental k^, 2) the optimum k^, and 
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3) the minimum mean-square error. The results of these 
theoretical studies are presented in Figures III-l through 
III-7 . 

Figure III-l shows the variation of the minimum value 

of the mean-square error with noise power while holding the 

signal power density fixed. With zero noise the minimum 

2 

mean-square error is 3.784 volt , while with infinite 

noise the minimum mean-square error is 90.275 volt^. For 
_2 

N q = 8.56x10 the noise contributes about half the error 
in the output of the optimum system. 

Figure III-2 demonstrates the range of the kj with 
varying noise power density for fixed signal power. It is 
to be noted that the k^ go to zero at infinite noise, so 
that the maximum possible error is when the system is 
completely silent. Such a range in the values of the k^ 
with changes in noise power density indicates that an 
adaptive system might be worthwhile even for this simple 
case. 

If we pick the values of k^ that are optimum for one 
noise level as fixed coefficients of our filters, it is 
possible to demonstrate the effect of a fixed filter 
operating on an input of fixed signal power but varying 
noise power density. Such a comparison is presented in 
Figure III-3. We see that for a plot of the mean-square 
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error of each of the fixed filters versus noise power 
density, the mean-square error curve is tangent to the curve 
of the minimum error at the point for which the k^ of the 
fixed filter are optimum, but that when the noise changes 
from the value for which the k^ are optimum, the fixed 
filter error curve diverges from the minimum error curve. 

It should be noted here that for the simple case being 
considered, it is possible to obtain excellent results by 
the simple expedient of building a switch which selects one 
of the three sets of coefficients. While such a procedure 
is obviously more economical than an adjustment procedure 
in this particular case, it must be remembered that this 
is but a small part of the total system, and that addition 
of more filters would increase the number of possible 
combinations of the k^ to the point that manual switching 
would be out of the question. 

For the case studied here, the regression surface 
being searched is a paraboloid when mapped using k^ k 2 , 
and e^ in a right-handed coordinate system. In order that 
the reader might observe the effect of noise power density 
on the shape of this surface, three graphical displays are 
made here. Figures III-4 and III-5 show the locus of points 
on the surface at planes of constant mean -square error for 
N q = 8.56xl0 -3 and 8.56xl0“ 2 respectively. Figure III-6 
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- 1. 




Figure III-5. e2 vs. and k 2 for N Q = 8.56xl0“ 2 , S o = 0.575 
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_2 

Figure III-6. Regression surface for N Q = 8.56x10 , S G = 0.575 
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shows the three dimensional configuration of the surface in 
the vicinity of the minimum for N 0 = 8.56xl0~ 2 . For both 
noise levels it is noted that the planes of constant 
intersect the regression surface in ellipses. It is 
possible to determine the equation of the major axis 

I 

(hereafter referred to as the line of minima) of these 
ellipses in terms of the k^ by algebraic manipulation of 
(2) . These equations are found to be 

k x = 4.704 - 1.005 k 2 for N 0 = 8.56xl0~ 3 and 
ki = 3.213 - 1.036 k 2 for N c = 8.56xl0 -2 . 

All of the theoretical cases considered above have 

been for S Q = 0.575 volt 2 sec. The variation of S Q from 

this value was also studied for N = 8.56xl0 -2 . The effect 

o 

on minimum mean-square error and the k^ is shown in Figure 
III-7. We see that the effect on the error in this region 
for small pertubations of S c from its standard value is 
not too great. 

The data obtained from these theoretical studies were 
used to define the convergence area in terms of k]^ and k 2 
so that convergence times could be obtained as outlined in 
Chapter II. The computer program was also used to obtain 
the value of e^ based on the k^ for experimental runs. The 
results of the digital computor calculations are presented 
in this section as well as in the next section. 
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B. EXPERIMENTAL RESULTS 



A study of the feedback loop integrator gain, D, as it 
affects the convergence time is presented in Figure III-8. 
(For this figure and those following A' = 1, S Q = 0..575, 
and N Q = 8.56x10 unless otherwise noted.) For values of 
D greater than 10 the system failed to keep the error 
within the convergence area for any consistent period of 
time. Further, convergence was not necessarily obtained at 
all. For runs with the gain less than 10 it was found 
that there was at most one overshoot of the convergence 
area and that it was in the initial phase of the conver- 
gence. On entering the convergence area after the initial 
overshoot, if any, it was found that the error remained 
within the convergence area throughout the remainder of the 
run. 

An example of this overshoot is shown in Figure III-9. 
The initial settings of the kj[ were = ^2 = 12 (hereafter 
the notation (ki,k 2 ) will be used) with D = 1. Note that 
the line of minima was reached in about the same time as that 
taken by the system shown above starting at (6,6) to reach 
the convergence area. 

The variation of mean convergence time with A‘ for D=1 
is shown in Figure III-10. For all of these runs the 
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Pigure III-8. Convergence time versus integrator gain. 

r ' •* 
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Figure 111-9* Typical convergence of the k^(with overshoot)* 
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Figure III-lOo Convergence time versus A ' 0 
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initial values were (6,6). The path followed from the ini- 
tial point to entry into the convergence area was essentially 
the same in all cases (i.e. direct to the convergence area 
on a line nearly perpendicular to the line of minima) . In 
no case did the system fail to converge. 

In figure III-ll the mean convergence time is shown 
for several initial values of the k^ . The line of minima 
and the convergence area are also shown. In all cases the 
initial movement was nearly perpendicular to the line of 
minima. Large overshoot of the convergence area occurred 
from (-6,-6) and a slight overshoot of the line of minima 
occurred from (-6,0). For all of the runs shown on this 
figure the integrator gain, D, was two since this value 
permitted convergence times on all runs within 48 seconds, 
the maximum time allowed if a(t) is to be fully utilized. 

A demonstration of the results of the convergence 
procedure over a 30 db range of noise power density is 
shown in Figures III-12 and III-13. S Q = 0.79 volt^sec 
was used for the runs in both these figures. The values 
of k^ and k 2 to which the experimental runs converged and 
the theoretical values of the k^ over the 30 db range of 
noise power density are shown in Figure III-12 » The 
minimum mean-square error, the mean-square error calculated 
by the method of the preceding section for the experimental 
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Figure XIX-/ 1 Convergence times from various initial settings 
of k.^ and k 2 . 52 
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kj^ based on the assumed spectra, and the experimentally 
measured mean-square error are shown in Figure III-13. 

Some saturation was encountered in the input K5U for the 
data point having the largest noise density, but this could 
be removed by linear attenuation of the input signal. 

Figures III-14 and III-15 are typical recordings show- 
ing the convergence of the k^ versus time. Values of the 
k^ taken from these figures demonstrate the movement of the 
error to the line of minima and then into the convergence 
area when plotted as in Figure III-9. 

Figures III-16 through III-24 show the mean-square 

error and the parameter k,(in this case k represents 

^ 2 

the coefficient of filter 2) as a function of time for the 
four filter case. In order to demonstrate both the range 
of the mean-square error and the fact that the system error 
converges to about the same value for several conditions 
of internal parameter settings, two scales are used on the 
recordings of the mean-square error. The initial squared 
error is integrated from -50 volts for 0.1 seconds. After 
the error has reached a value near its minimum, the squared 
error is amplified by a factor of four and integrated from 
0 volts for 0.1 seconds. The apparent discontinuity in the 
recording indicates the change of the initial value of the 
integrator, and the increased amplitude in the integrated 
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squared error a short time later indicates the introduction 
of the factor of four amplification. The results of several 
selected runs are shown in Table III. 

TABLE III 



RESULTS OF SELECTED RUNS WITH FOUR FILTERS 

Initial Value Final Value Final Mean- Run 



D 


il' 


*2* 


k 3 


k 4 








k 4 


Square Error 


Number 






For 


the following 7 


runs 


N o 


= 8.56xl0" 2 




2 


30 


30 


30 


30 


-2 


10 


7 


-2 


28.5 


4F1 


2 


0 


0 


0 


0 


-3 


19 


-2 


2 


26.7 


4F2 


2 


-30 


-30 


-30 


-30 


0 


1 


34 


-27 


32.0 


4F3 


2 


-30 


-30 


30 


30 


-3 


15 


5 


-3 


25.0 


4F4 


2 


30 


30 


-30 


-30 


-7 


21 


-19 


13 


25.0 


4F5 


5 


30 


30 


30 


30 


-10 


36 


-22 


14 


25.0 


4F6 


.5 


30 


30 


30 


30 


0 


10 


4 


3 


25.0 


4F7 






For 


the following 7 


runs 


No = 


8 . 56x10 


-3 




2 


30 


30 


30 


30 


-15 


31 


3 


3 


9.9 


4F8 


2 


0 


0 


0 


0 


-15 


30 


9 


0 


11.1 


4f9 


2 


-30 


-30 


-30 


-30 


-12 


10 


42 


-25 


9.9 


4F10 


2 


-30 


-30 


30 


30 


-9 


0 


19 


6 


8.7 


4F11 


2 


30 


30 


-30 


-30 


-21 


48 


-3 


-4 


9.9 


4F12 


5 


30 


30 


30 


30 


-22 


50 


-16 


6 


11.1 


4F13 


.5 


30 


30 


30 


30 


-9 


17 


9 


9 


9.9 


4F14 



b<{ 0*2 

NOTE: Multiply k| A to get k^. 
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It was found that about 45 minutes were required to 
set up the computer for each set of runs. The largest 
portion of this time was spent in adjusting the MU/DV units 
for the proper value of their division constant and removing 
the mean DC level from them. 

Removal of the average DC level from the K5M units 
was also necessary. It was found that 0x0/50 / 0x0/20 
and that 0x50/50 / 0x50/20 / 0x20/50 for operations with 
the K5M units . In view of the extreme ranges encountered 
in conducting the tests discussed above, this was particu- 
larly disconcerting. The values involved were small, on 
the order of 0.035 volts rms, but when these are integrated 
for a period of several seconds, they begin to contribute 
an error about which little can be done. 

C. DISCUSSION OF RESULTS 



In considering the effect of integrator gain, D, on 
convergence times when A’ = 1, we first examine both 
extremes in gain settings. It is easily seen that for an 
integrator gain of zero, convergence can never be obtained, 
and we expect that small values of gain setting will lead 
to slow convergence. At high gains, the 50 volt upper limit 
of the computing equipment enters the problem in two ways . 
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First, a(t) will be at a constant value (saturation) for 
t D/50 - 2, D •> 100. Second, since the function A't + 2 
reaches its maximum value at t = 48 seconds for A' =1, 
the minimum value which a(t) can reach is D/50. Both of 
these effects violate the requirement that a(t) decrease 
monotonically . The first effect means that the convergence 
procedure does not start to settle down until some time 
greater than zero, and the second effect may result in 
undamped oscillations if a near steady condition has not 
been reached by the system prior to the time a(t) reaches 
its minimum value. In any event, we desire that a(t) be as 
near zero as possible for t greater than some T 0 and that 
there be no initial saturation. Both of these conditions 
are met for small values of A'. In general, we expect 
that there will be some value of D which will minimize 
the time to convergence, and that there will be some range 
of settings of D about this minimum which will give suitable 
values of convergence time. 

From Figure III-8 it is seen that an asymptotic approach 
to infinite convergence time is obtained for integrator 
gains approaching zero. Minimum convergence time is obtain- 
ed for an integrator gain setting of two. Convergence time 
is found to remain nearly constant for integrator gains 
between four and seven. Further, it was observed that for 
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values of gain less than two, the system converged without 
any overshoot of the convergence area. However, for set- 
tings of D greater than two, overshoot became increasingly 
severe. Thus, the minimum time to convergence is obtained 
for a setting of D just less than that value which first 
results in oscillations. 

The combined effect of the shape of the regression 
surface and the weighting of a(t) shown in Figure III-9 
is perhaps the best graphical portrait of the convergence 
process. The initial movement of the error perpendicular 
to the line of minima shown here is typical. There is an 
overshoot of the convergence area, but the system corrects 
itself and turns back toward the line of minima. As the 
error decreases while following the line of minima into 
the convergence area, we see the increasingly slower move- 
ment which is also characteristic of the system. 

For the experimental system studied in this work the 
convergence time is less than one second for A' •<. 2.5. 

When the product A't c (where t c = convergence time) is less 
than 0.2, we expect this product will be dominated by the 
constant 2 in A't + 2, and that there will be no further 
change in the convergence time for decreasing values of A’ . 
This effect is shown in Figure III-10 where the curve of 
convergence time versus A' has a flat portion for A' <0.5. 
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For each combination of noise power density and signal 
power density there is a fixed value of K Q . Since it is 
rejuired that 0 ^ A' ^ 4 K q when D = 1, it is expected 
that increasing A' beyond A' = 4K 0 will result in increas- 
ing amounts of time being taken for the system to converge. 
Although K Q was not computed for this experimental system, 
it is noted that for 10 > A' >■ 2 the convergence time 
increases linearly with A' . Since oscillations were absent 
throughout the range 0 < A' -<T 10, we may conclude that the 
convergence procedure is not overly dependent on A' when 
convergence occurs prior to the time that a(t) reaches its 
minimum value. 

From the convergence times shown in Figure III-ll it 
is seen that the system takes very little time to reach the 
line of minima while taking the majority of the convergence 
time to move down the valley along the line of minima. 

For instance, if we look at the two points (-6 6) and 
(0,6), we see that the convergence time is about the same 
from both points, but that a slightly longer time is taken 
by the movement from the initial setting furtherest from 
the line of minima. Looking at the two points (6,0) and 
(6,-6) we see that the perpendicular distance to the line 
of minima is about the same, but that for the initial setting 
furtherest from the convergence area, the convergence time 
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is about a third longer compared with the nearer initial 
setting. 

The seeming deviation from this analysis is present 
if one considers the points (6,6) and (-6,-6). Here the 
discrepancy is due to an overshoot when starting from 
(-6,-6) which does not occur when starting from (6,6). 

After the overshoot occurs the system returns to the line 
of minima at about (10,-8) and the excessive time is spent 
in transit down the line of minima from that point. After 
a slight overshoot of the line of minima from the initial 
position of (0,-6) ; the system returns to the line of minima 
only a short distance from the convergence area. Prom 
the foregoing discussion we again see that the system 
giving the most rapid convergence results when the gain 
settings are so adjusted that the system is just on the 
verge of overshooting the convergence area. Moreover^ the 
range in convergence times from 0.45 to 38.5 seconds is 
not indicative of the best possible performance of the 
system since all of these runs were obtained with that 
value of D and A' which gave minimum convergence time from 
( 6 , 6 ). 
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The information shown in Figures III-12 and III-13 
demonstrate that this system is capable of operating 
properly over a range of 30 db of noise. Thus the system 
is suitable for the proposed application in an adaptive 
filter for radio communications. 

From Table III and Figure III-l it is seen that in the 
four filter case, the error on completion of optimization 
is about 40% less than that obtainable with two filters. 
This is to be expected since with more third order filters , 
we can more closely approximate the second order desired 
signal. 

A computer characteristic which may have introduced 
small inaccuracies in the experimental results was the 
inability to adequately compensate for the input dependent 
DC level of the multipliers . Admittedly this level was 
small, but it can become quite troubelsome when the noise 
power density and/or the signal power density changes from 
one trial to the next as must be the case if this system 
is to be useful in the adaptive configuration. For this 
work we could adjust the DC level to a mimimu... at the 
optimum settings of the k^ so as to avoid drift after the 
system had reached its near optimum position. Such an 
adjustment is possible only because we knew a priori 
what the optimum settings were. An adaptive system is, 
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of course, unnecessary if we have such knowledge of the 
optimum settings. A way of avoiding this problem would be 
to heterodyne all signals up about some suitable carrier 
(say 10 Kc.) and work with the heterodyned signals. 
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CHAPTER IV 



CONCLUSIONS AND RECOMMENDATIONS 

A. CONCLUSIONS 

We have shown that the adjustment procedure does 
produce the desired results for the mean-square error 
criterion. The time taken to converge can be controlled 
by adjustment of the feedback loop gain. Since conver- 
gence times of well under 30 seconds can be obtained with 
this system and the system does perform satisfactorily over 
the required 30 db range in noise power density, it is 
applicable to use in an adaptive filter system for 
radio communications. Further, it has been shown that 
the overall gain in performance over a fixed filter 
which can be achieved by even this simple example is 
of the order of 5 to 10 db at the extreme deviations of 
the noise power density from the fixed filter design value. 

B. RECOMMENDATIONS 

Although the adjustment procedure worked well here, 
it should be pointed out that the cost of stabilized multi- 
plier units might make such a system impractical from an 
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economic viewpoint in the use proposed here. However, if 
use of this unit materially decreased the probability of 
enemy detection of a 25 million dollar ship, the expendi- 
ture of several thousands of dollars would be well justified. 
The possibility of reducing the cost of operational units 
from that of the highly specialized equipment used for 
this experimental work is great. For instance, further 
investigation of applications of this adjustment procedure 
might look into the possibility of heterodyning all signals 
about a suitable carrier and working with the heterodyned 
signals. This would avoid the DC drift problem encountered 
here and may permit the use of relatively inexpensive 
units for the multipliers as well as for the entire system. 

The design engineer having access to existing analog 
computing equipment should consider this adjustment 
procedure as it offers reasonable savings in filter design 
time over that required by present statistical measurement 
and computational techniques. 
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